There is no universal source code for an infinite source alphabet
نویسندگان
چکیده
We show that a discrete infinite distribution with finite entropy cannot be estimated consistently in information divergence. As a corollary we get that there is no universal source code for an infinite source alphabet over the class of all discrete memoryless sources with finite entropy.
منابع مشابه
Universal Source Codes Lecturer : Himanshu Tyagi Scribe : Sandip Sinha
• Huffman Code (optimal prefix-free code) • Shannon-Fano code • Shannon-Fano-Elias code • Arithmetic code (can handle a sequence of symbols) In general, the first three codes do not achieve the optimal rate H(X), and there are no immediate extensions of these codes to rate-optimal codes for a sequence of symbols. On the other hand, arithmetic coding is rate-optimal. However, all these schemes a...
متن کاملPattern Coding Meets Censoring: (almost) Adaptive Coding on Countable Alphabets
Adaptive coding faces the following problem: given a collection of source classes such that each class in the collection has non-trivial minimax redundancy rate, can we design a single code which is asymptotically minimax over each class in the collection? In particular, adaptive coding makes sense when there is no universal code on the union of classes in the collection. In this paper, we deal...
متن کاملPrediction of Large Alphabet Processes and Its Application to Adaptive Source Coding
The problem of predicting a sequence x1, x2, · · · generated by a discrete source with unknown statistics is considered. Each letter xt+1 is predicted using the information on the word x1x2 · · · xt only. This problem is of great importance for data compression, because of its use to estimate probability distributions for PPM algorithms and other adaptive codes. On the other hand, such predicti...
متن کاملGrammar-based codes: A new class of universal lossless source codes
We investigate a type of lossless source code called a grammar-based code, which, in response to any input data string over a fixed finite alphabet, selects a context-free grammar representing in the sense that is the unique string belonging to the language generated by . Lossless compression of takes place indirectly via compression of the production rules of the grammar . It is shown that, su...
متن کاملUniversal Source Coding of Finite Alphabet Sources
The application of Shannon’s rate distortion theory to real data compression practice has been limited by a lack of effective distortion measures for most real users and incomplete statistical knowledge of real sources. Universal source coding techniques attempt to overcome the latter limitation by developing codes that are effective (in the sense of approaching the theoretical rate distortion ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- IEEE Trans. Information Theory
دوره 40 شماره
صفحات -
تاریخ انتشار 1994